The first thing we need to do is get scripts and files onto the
server. You can do this with sftp. You can view a tutorial
of using sftp here.
I’ll show you how it works on my AWS server.
cd command or you can put the files you
want to move in your home directory - this is where the terminal will be
by default.sftp root@<url> where
<url> is the url of your Digital Ocean droplet. In my
instance, I needed to pass the private key, but you don’t have to do
that, you’ll have to give a passphrase to login. You should see
something like this:Notice how the command prompt starts with sftp>. This
isn’t something you type, it’s just the terminal’s way of telling you
that you’re connected using sftp. Next, you’ll need to put files on the
server. To see what’s in your local directory, you can type
lls and to see what files are currently on the server, you
can type ls.
As you can see from the output above, there are a bunch of files
locally and none currently on the server. To put the files on the
server, say sacl_mrp_script_v2.R, we would use
put.
put sacl_mrp_script_v2.R
You will need to do this with all files that get called by your
script. If all the files you need get read in from external URLs, then
this is the only file you need to upload. Once the files have been
successfully transferred, you can type exit to leave the
sftp session. The command prompt should change back to the Windows
prompt.
Next, you can ssh into the server. You should be able to do this with:
ssh root@<url>
where <url> is the url of your Digital Ocean
droplet. You should see something like this:
The command prompt name should be the same as name of the droplet.
Next, you need to make sure that all the packages you’re using are
installed in R. You can do this by typing R at the command
prompt and you will see the R console.
There, you can install any packages using
install.packages('pkgname') just like you would in any R
session. Once you’ve done that, you can quit the R window by tying
quit('no'). The next step is to run the R script. The trick
here is that you will want to do it in such a way that the server keeps
executing the script when you get disconnected or log out. If you just
run it at the terminal, this won’t be the case. The trick here is to use
screen. There is a little tutorial on how it works here. The
basic idea is that you start a screen session, run your script, then
detach from the screen session. The script will keep running in the
background. When you log back in, you can re-attach to the screen
session and see the output of your script. At the command prompt, you’ll
type screen and hit enter. This will send you to a new
screen giving some information about the authors. You will need to hit
enter again to get past this screen.
Once you’ve done this, you will see the command prompt again. You can now run your script with:
R CMD BATCH sacl_mrp_script_v2.R
All output will be sent to a file called
sacl_mrp_script_v2.Rout. If your file name doesn’t end with
.R, for example if your file was
sacl_mrp_script_v2.txt, then the output would be the full
file name (with extension) followed by .Rout, like
sacl_mrp_script_v2.txt.Rout. I’m going to show you how this
works using a simple script that just prints a new number every 5
seconds. It’s called long_script.R and its output will be
sent to long_script.Rout. In the gif below, you’ll see me
get into a screen, set the script running and then detach from the
screen. Detaching is done by holding down the ctrl key and
then typing a followed by d. You should see a
message that you’ve detached from the screen. Then, I’ll use
cat to show the R output.
When the screen is detached, you can type
cat sacl_mrp_script_v2.txt.Rout any time you want to see
the progress. You can always reattach the screen with
screen -r. The gif below shows me doing that. Notice that
when I do that, the cursor is blinking, but there is no command prompt
there. If that’s the case, the script is still running. If you want to
stop if from running, you can type ctrl + c,
but that will stop the script and you will have to start it again from
scratch.
If you want to know if the script is still running, you can type
top at the command prompt. That will show you the processes
that are running. If you rscript is still running, you will see
R show up using up resources, like it is at the top of the
output below.
You can exit from the top output by typing
ctrl + c.
If your script finishes running, when you reattach the screen, you
should see the cursor at a command prompt. When you do ls,
you should see the files that you had written out when the script
completed.
The process of running the MRP estimates is exactly the same as
above, but with a different script. You will need to upload the script
and any files it calls using sftp. Then, you can ssh into
the server, start a screen session, run the script with
R CMD BATCH scriptname.R and then detach from the screen.
You can check on progress by reattaching to the screen or by looking at
the output file with cat scriptname.Rout. When the script
is done, you can reattach to the screen and you should see a command
prompt. You can then use ls to see if your output files are
there.
Some things to think of.
.rds files) are on the
server and that they’re being called by the appropriate filenames in the
script.After the estimates are done, you can go to the server and open R.
You can read in the constituency estimates and do any aggregation or
plotting that you want. You can write out any files you want to your
home directory. You can then use sftp to get those files
back to your local machine using the get command that works
just like put.